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I.  Summary:  Objectives  and  Status  of  Effort 


In  this  report  we  summarize  our  accomplishments  under  the  research  program 
supported  by  Grant  F49620-98-1-0349.  The  basic  scope  of  this  research  program  is  to 
carry  out  fundamental  research  in  several  interrelated  areas:  (a)  the  use  of  multiresolution 
methods  in  statistically  optimal  image  analysis,  with  applications  in  multisensor  fusion, 
image  segmentation,  anomaly  detection,  and  optimal  processing  in  the  presence  of  speckle; 
(b)  the  blending  of  sensor  physics  and  statistical  models  for  computationally  efficient,  near- 
optimal  inversion  and  image  formation  with  applications  in  radar  imaging  and  other  areas  of 
interest  to  the  Air  Force;  (c)  the  development  of  statistically  robust  and  computationally 
efficient  nonlinear  image  processing  methods  with  applications  in  segmentation,  edge 
detection,  and  feature  extraction  for  object  recognition;  (d)  the  development  of 
multiresolution  and  wavelet-based  methods  for  robust  feature  extraction,  with  applications 
in  object  recognition;  (e)  the  development  of  fast  numerical  methods  for  a  number  of 
difficult  problems  in  statistical  image  processing  using  a  blend  of  advanced  numerical 
methods  and  our  work  on  multiresolution  algorithms;  and  (f)  the  extension  of  our 
multiresolution  modeling  framework  to  include  very  different  granularities  of  information, 
from  image  pixels  to  discrete  variables  representing  context,  with  applications  to  the 
emerging  areas  of  global  awareness  and  integrated  spatial  databases.  Key  features  of  this 
proposal  are  that  (i)  it  blends  together  methods  from  several  fields — statistics  and 
probabilistic  modeling,  signal  and  image  processing,  mathematical  physics,  scientific 
computing,  Bayesian  networks,  and  nonlinear  differential  equations-to  produce  new 
approaches  to  emerging  and  challenging  problems  in  signal  and  image  processing;  (ii)  it 
both  builds  on  the  research  results  we  have  obtained  under  our  current  grant  and  also 
explores  new  directions  in  which  our  approaches  appear  to  have  significant  merit;  and  (iii) 
each  aspect  of  the  proposed  program  contains  both  fundamental  research  in  mathematical 
sciences  and  important  applications  of  direct  relevance  to  Air  Force  missions. 

The  principal  investigator  for  this  effort  is  Professor  Alan  S.  Willsky.  Professor 
Willsky  has  been  assisted  in  this  effort  by  Dr.  John  Fisher,  a  research  scientist  in  MIT's 
Laboratory  for  Information  and  Decision  Systems,  and  by  several  graduate  research 
assistants  as  well  as  additional  thesis  students  not  requiring  stipend  or  tuition  support  from 
this  grant.  In  the  next  section  we  describe  our  research  accomplishments;  in  Section  HI  we 
indicate  the  individuals  involved  in  this  effort;  in  Section  IV  we  list  the  publications 
supported  by  this  effort;  and  in  Section  V  we  discuss  interactions  and  transitions. 


II.  Accomplishments/New  Findings 

In  this  section  we  briefly  describe  the  research  accomplishments  we  have  achieved 
with  support  provided  by  this  grant.  We  limit  ourselves  here  to  a  succinct  summary  and 
refer  to  the  publications  listed  at  the  end  of  this  report  (as  well  as  the  previous  progress 
reports)  for  detailed  developments. 

2.1  Multiresolution,  Hierarchical,  and  Relational  Modeling 

The  research  described  in  this  section  is  developed  in  great  detail  in  a  number  of 
papers  and  reports  [1,3,6,17,19,22,27,33,34,41,44,60-63].  The  overall  objective  of  this 
portion  of  our  research  is  the  development  of  methods  for  constructing  stochastic  models 
for  phenomena  that  vary  over  space,  time,  and  hierarchy  and  that  possess  structure  which 
can  be  exploited  to  construct  efficient  and  scaleable  algorithms  for  statistical  inference  (the 
subject  of  subsequent  sections  of  this  report). 

a)  As  we  described  in  our  earliest  work  on  multiresolution  models  on  trees,  each 
node  on  the  tree  in  a  multiresolution  model  separates  the  tree  into  several  disjoint  subtrees 
(corresponding  to  the  disconnected  components  of  the  tree  if  that  particular  node  were 
removed).  The  role  of  the  state  at  that  node  is  to  act  as  a  statistical  interface  between  these 
subtrees.  That  is,  conditioned  on  the  state  at  that  node,  the  sets  of  variables  in  these 
disjoint  subtrees  are  mutually  uncorrelated.  In  our  early  work  in  this  area  we  combined  this 
characterization  of  state  with  a  generalization  of  the  notion  of  canonical  correlations  to 
develop  a  first  approach  to  constructing  multiscale  models  that  yield  statistics  for  the  finest- 
scale  process  (i.e.,  the  variables  at  the  fmest-scale  leaf  nodes  of  the  tree)  that  approximately 
match  those  of  a  given  stochastic  process  or  random  field.  This  approach  to  building 
multiresolution  models,  while  providing  excellent  models  for  stochastic  phenomena  that  we 
had  not  been  able  to  model  previously,  had  significant  limitations  and  drawbacks,  many  of 
which  have  been  exposed  in  our  recent  work  [34].  First  of  all,  the  use  of  canonical 
correlations  as  the  criterion  for  determining  state  variables  at  each  node  is  extremely 
complex  computationally.  Secondly,  the  use  of  the  criterion  of  maximal  decorrelation  is 
not  necessarily  the  right  one  to  use.  In  particular,  as  argued  in  [34],  the  reason  for 
constructing  multiresolution  models  is  that  they  then  provide  the  basis  for  very  efficient 
statistical  inference  computations,  e.g.,  for  the  calculation  of  optimal  estimates  of  the 
phenomenon  being  modeled  given  noisy  and  possibly  incomplete  observations.  This 
suggests  that  perhaps  a  better  criterion  for  designing  a  state  variable  at  a  node  on  a  tree  is  to 
choose  states  that  do  the  best  job  of  estimating  variables  at  the  leaf  nodes  of  one  of  the 
subtrees  given  measurements  of  the  leaf  nodes  on  another  of  the  subtrees.  Not  only  does 
this  approach  lead  to  the  definition  of  state  variables  designed  in  a  manner  more  consistent 
with  the  ultimate  use  to  which  the  model  will  be  put,  but  it  also  leads  to  a  dramatic 

reduction  in  computational  complexity. 

Moreover,  in  [34]  we  also  provide  a  deeper  interpretation  of  what  the  state  variables 
are.  In  particular,  in  its  most  general  and  unconstrained  form,  the  states  in  a 
muitiresolution  model  at  all  but  the  leaf  nodes  are  simply  hidden  variables,  designed  simply 
to  fulfill  their  role  in  statistically  decorrelating  the  subtrees  extending  away  from  each  node. 
However,  in  many  cases  we  want  these  variables  to  have  real  physical  significance,  namely 
that  they  truly  do  represent  coarser  versions  of  the  finest  scale  process.  Adopting 
terminology  from  state  space  realization  theory  for  time  series,  a  multiresolution  model  in 
which  each  state  consists  of  a  set  of  nonlocal,  coarser  functionals  of  the  finest-scale 
process-is  called  an  internal  model.  In  [34]  we  provide  a  complete  theoretical 
characterization  of  the  properties  of  states  of  internal  models  which,  in  particular,  shows 
that  these  states  have  a  scale-recursive  structure  from  fine-to-coarse.  That  is,  much  as  in 


wavelet  analysis,  each  state  in  an  internal  multiscale  model  consists  of  nonlocal  functionals 
of  its  immediate  descendent.  In  [34]  we  develop  an  algorithm  that  not  only  produces 
internal  stochastic  models  but  that  exploits  the  fine-to-coarse  characterization  of  internal 
states  to  achieves  even  greater  computational  efficiency.  The  bottom  line  is  that,  while  our 
previous  algorithm  had  0(N* )  complexity,  where  N  is  the  number  of  leaf  nodes  in  the  tree 
(e.g.,  the  number  of  pixels  in  the  image  of  interest),  our  new  algorithm  has  complexity  that 
has  0(N2)  complexity. 

b)  The  second  component  of  this  portion  of  our  research  involves  the 
rapprochement  of  our  multiresolution  methods  with  the  machinery  of  wavelet  analysis  [22]. 
This  is  nontrivial  intellectually,  as  it  provides  a  different  way  in  which  to  view  wavelet 
functionals.  In  particular,  while  wavelets  are  usually  thought  of  as  a  tool  for  analyzing 
signals  and  images,  we  have  now  shown  how  we  can  incorporate  some  of  the  machinery 
of  wavelets  into  our  multiresolution  framework  which  focuses  on  modeling  signals  and 
images.  In  particular,  by  making  this  embedding,  our  statistical  methodology  naturally 
leads  to  a  very  different  method  for  coarse-to-fine  signal  modeling-namely  statistically 
optimal  coarse-to-fine  prediction-than  that  suggested  by  standard  wavelet  synthesis. 
Combining  this  with  the  theoretical  concept  of  an  internal  multiresolution  model,  we  have 
shown  that  we  can  obtain  remarkably  accurate  models  of  fractal  processes  with  very  low- 
order  wavelets-indeed  far  lower  order  than  is  achieved  by  standard  wavelet  methods. 
Moreover,  since  the  focus  of  our  methodology  is  first  on  building  models  rather  than  on 
analyzing  data,  we  can  directly  use  these  same  models  to  solve  problems  which  are  not 
easily  or  naturally  considered  within  the  usual  wavelet  framework-e.g.,  statistical 
reconstruction  based  on  sparse  and  irregularly  sampled  data,  optimal  fusion  of 
multiresolution  measurements,  etc. 

c)  In  the  third  portion  of  our  research  in  this  area  (described  in  detail  in  [41),  we 
have  developed  a  major  new  result  that  addresses  a  significant  issue  not  considered  in  our 
previous  work  on  building  multiscale  models.  In  particular,  all  of  our  other  work  on 
multiscale  modeling  assumes  that  we  have  available  the  entire  covariance  of  the  fine  scale 
process  to  be  modeled.  In  many  applications  this  is  not  reasonable  for  two  reasons:  (i)  for 
large  fields  we  may  not  know  the  entire  covariance,  i.e.,  the  covariance  between  every  fine- 
scale  pair  of  points  in  the  field  being  modeled;  and  (ii)  even  if  we  did  know  this  entire 
covariance,  explicitly  storing  or  using  it  in  a  realization  procedure  quickly  becomes 
prohibitive  for  large  problems.  Thus  we  have  looked  at  the  problem  of  building  a 
multiscale  model  (which  implicitly  specifies  the  entire  covariance)  directly  from  knowledge 
of  only  a  part  of  the  fine-scale  covariance  matrix.  This  turns  out  to  be  a  novel  variant  of  the 
intensely  studied  problem  of  covariance  extension. 

In  our  work  we  have  looked  at  what  starts  out  as  a  standard  problem:  we  are  given 
a  diagonal  band  of  the  covariance  matrix.  What  is  well-known  is  that  the  max-entropy 
extension  of  this  covariance  is  a  Markov  process  of  order  equal  to  the  width  of  the  specified 
covariance  band,  and  the  celebrated  Levinson  algorithm  allows  one  to  specify  not  only  this 
extension  implicitly  through  a  time-dynamic  model  but  also  to  characterize  all  possible 
extensions  in  terms  of  so-called  partial  correlation  coefficients.  This  algorithm  allows  one 
to  compute  missing  covariance  elements  band-by-band,  extending  outward  from  the 
specified  band.  What  we  have  developed  is  a  significantly  nontrivial  extension  of  this  idea 
that  involves  filling  covariance  elements  in  very  different  orders.  The  motivation  for  this 
extension  is  our  earlier  work  that  showed  that  Markov  processes  can  be  modeled  exactly  on 
a  multiscale  tree.  Constructing  the  parameters  of  this  multiscale  model  requires  knowledge 
of  certain  elements  of  the  overall  covariance  that  do  not  at  all  fall  in  a  banded  structure  but 
rather  require  filling  in  a  fractal  pattern  of  elements  in  the  covariance  matrix.  In  seeing  if 
one  could  indeed  do  this  efficiently,  we  developed  new  graph-theoretic  results  that 
characterize  sequential  orders  in  which  covariance  elements  can  be  computed  efficiently  and 
to  which  Levinson’s  algorithm  can  be  extended.  We  then  also  show  that  the  elements 


required  to  be  filled  to  construct  our  multiscale  model  indeed  satisfy  the  conditions  of  our 
graph-theoretic  result,  allowing  us  to  build  multiscale  models  efficiently.  We  believe  that 
this  result  is  quite  significant,  not  only  because  of  its  contribution  to  a  very  challenging  and 
widely  studied  theoretical  problem  but  also  because  the  graph-theoretic  machinery  we  have 
developed  opens  up  many  interesting  and  important  research  problems. 

d)  A  broad  area  for  the  extension  of  our  multiresolution  methodologies  is  to  non- 
Gaussian  and  nonlinear  models.  In  [44, 61-63]  we  describe  some  of  our  recent  work 
motivated  by  the  statistical  behavior  of  wavelet  coefficients  of  real  imagery.  In  particular, 
while  wavelets  are  known  to  do  a  good  job  of  decorrelating  images,  the  resulting  wavelet 
coefficients  are  generally  neither  Gaussian  nor  independent.  In  particular,  the  empirical 
distributions  of  these  coefficients  generally  show  heavy-tailed  behavior  with  sharp  peaks  at 
zero.  Moreover,  there  are  characteristic  dependencies  among  “neighboring”  wavelet 
coefficients  (neighbors  in  location,  scale,  or  orientation  for  steerable  2-D  wavelet  pyramids) 
in  which  if  one  coefficient  is  large  there  is  a  higher  probability  that  its  neighbors  are  as 
well.  It  is  precisely  this  type  of  behavior  that  is  exploited  in  wavelet-based  compression 
algorithms  such  as  embedded  zero-trees.  Motivated  by  these  empirical  observations,  we 
have  begun  to  develop  a  framework  for  capturing  what  we  refer  to  as  “wavelet  cascades” 

which  capture  both  a  variety  of  heavy-tailed  distributions  including  a-stable  processes  as 
well  as  the  self-reinforcing  cascade  of  large  wavelet  coefficients. 

The  model  class  described  in  [62]  captures  these  characteristics  with  a  construct  that 
has  a  great  deal  of  structure.  In  particular,  our  models  involve  the  use  of  general  Gaussian 
mixtures  as  a  method  for  generating  a  variety  of  heavy-tailed  distributions.  More  precisely, 
our  model  involves  the  multiplicative  mixing  of  a  linear-Gaussian  multiresolution  model 
with  a  memoryless  nonlinear  function  of  a  second  multiscale  Gaussian  process  (known  as 
the  “multiplier”  process).  By  adjusting  the  scale-to-scale  correlation  of  these  processes  and 
the  shape  of  the  memoryless  nonlinearity  we  can  model  a  surprisingly  rich  set  of  heavy¬ 
tailed  behavior  that  exhibit  the  types  of  cascade  phenomena  found  in  real  imagery.  Thanks 
to  the  structure  of  these  models,  we  believe  that  they  represent  fertile  ground  for  detailed 
analysis  and  algorithm  development.  Indeed  in  [62]  we  present  some  preliminary  resluts 
showing  how  this  structure  can  be  exploited  in  developing  efficient  estimation  and  fusion 
algorithms. 

e)  In  other  recent  work  [37]  we  have  initiated  an  effort  aimed  at  the  development  of 
nonparametric  statistical  methods  for  constructing  multiresolution  models  for  applications 
ranging  from  object  recognition  to  data  mining.  In  particular  the  construction  of  multiscale 
models  involves  two  distinct  steps,  the  identification  of  the  state  variables  to  be  used  at  each 
node  of  the  tree  and  then  the  identification  of  the  coarse-to-fine  multiscale  statistical  model. 
In  the  context  of  linear-Gaussian  models,  the  second  of  these  is  comparatively 
straightforward  and  is  roughly  comparable  to  the  one-step  prediction  problem  for  time 
series  (although  on  trees,  each  parent  node  has  several  children  requiring  prediction  steps). 
In  the  case  of  nonlinear  multiresolution  models,  both  of  these  steps  are  highly  nontrivial, 
and  in  our  initial  work  we  have  focused  on  the  second  one.  That  is,  in  our  work  to  date  we 
have  fixed  a  priori  the  variables  that  comprise  the  states  at  each  node  and  then  seek  to  build 
nonparametric  coarse-to-fine  statistical  models  relating  these  variables.  The  specific  context 
for  our  initial  work  in  this  area  is  SAR-based  ATR,  a  subject  described  in  subsequent 
sections  of  this  report.  What  is  of  significance  for  this  section  of  our  discussion  are  the 
novel  aspects  of  the  problem  of  building  nonparametric  models  for  the  coarse-to-fine 
predictive  model.  As  in  the  other  aspects  of  our  work,  the  fact  that  we  have  a  tree 
introduces  new  issues  not  encountered  for  time  series.  In  particular,  in  time  series 
analysis  stationarity  or  local  stationarity  is  invoked  in  order  to  obtain  some  stability  to  the 
estimates  of  the  time  series  model:  e.g.,  we  assume  a  stationary  distribution  for  successive 
data  points  and  thus  can  use  each  successive  pair  of  data  points  as  a  sample  from  the 


distribution,  providing  a  significant  number  of  samples.  What  types  of  stationarity  make 
sense  on  a  tree?  Across  nodes  at  the  same  scale?  Across  scale?  Furthermore,  we  now 
have  several  parent-child  data  pairs  with  the  same  parent  but  different  children.  Obviously 
these  are  not  independent  samples  since  they  share  the  same  parent,  but  each  such  pair  does 
contain  additional  information.  How  do  we  account  for  this?  In  our  initial  work  [37]  we 
have  both  posed  these  questions  and  taken  very  preliminary  looks  at  them,  yielding  models 
for  SAR  data  that  we  have  then  used  for  object  discrimination  with  sufficient  success  to 
make  clear  that  further  pursuit  of  these  issues  is  warranted. 

f)  The  motivation  for  this  portion  of  our  research  [7,10,21,42-43]  is  the  problem 
of  fusion  of  data  for  spatially-distributed  phenomena  or  imagery  that  also  evolves  in  time. 
The  challenge  in  such  problems  is  readily  seen  when  one  looks  at  their  time-recursive 
structure.  In  particular,  from  this  perspective,  the  problem  can  be  seen  as  one  of 
propagating  estimates  of  entire  spatial  random  fields  at  successive  instants  in  time  together 
with  the  propagation  of  models  of  the  uncertainty  in  these  estimates  (as  these  models  are 
then  needed  to  specify  the  statistically  correct  way  in  which  to  assimilate  data  in  the  future). 
Brute  force  application  of  standard  recursive  estimation  concepts  (e.g.,  like  the  Kalman 
filter)  lead  to  approaches  that  do  not  scale  at  all  well  with  domain  size  and  in  fact  become 
prohibitive  even  for  comparatively  small  spatial  domains.  The  perspective  we  have  adopted 
in  our  initial  work  is  to  develop  algorithms  that  directly  propagate  multiresolution  statistical 
models  for  estimation  errors  as  time  evolves  and  as  we  incorporate  new  data.  The  idea  here 
is  that  such  models,  which  specify  the  error  statistics  implicitly  (rather  than  explicitly  as  in 
the  standard  Kalman  filter),  admit  fast  algorithms  for  incorporating  new  data  as  we 
indicated  in  the  preceding  section.  The  key,  then,  is  finding  efficient  methods  for 
propagating  such  models  over  time.  . 

Our  initial  investigation  in  this  area  involved  the  study  of  1-D  space-time  problems, 
for  which  we  have  demonstrated  that  nearly  optimal  performance  can  be  achieved  with 
computational  load  per  time  point  that  grows  as  NlogN,  where  N  is  the  number  of  points  in 
the  spatial  domain.  This  compares  to  the  N3  complexity  per  time  point  for  standard  Kalman 
filtering.  The  key  innovation  here  involves  developing  a  method  for  propagating  a 
multiscale  model  for  the  spatial  uncertainty  in  a  random  field  across  time,  where  the 
temporal  dynamics  has  the  effect  of  “mixing”  scales  in  the  field.  Because  of  this  mixing, 
the  statistics  relating  variables  at  different  scales  changes,  and  computing  these  new 
statistics  and  then  calculating  the  scale-to-scale  dynamics  of  the  new  multiscale  model  for 
this  temporally-mixed  field  is  a  computationally  prohibitive  task  if  done  explicitly  without 
exploiting  any  structure.  We  have  been  able  to  overcome  this  computational  obstacle 
precisely  through  such  exploitation.  In  particular,  in  our  work  we  have  focused  on  models 
in  which  the  temporal  models  involve  two  fundamental  mechanisms:  transport,  in  which 
spatial  variables  are  in  essence  carried  along  by  a  flow  field,  and  diffusion,  in  which  there 
is  spatially  local  mixing  that  takes  place.  While  our  most  complete  demonstration  of  the 
effectiveness  of  this  method  is  for  problems  with  a  one-dimensional  spatial  domain,  our 
method  suggests  ways  in  which  to  extend  this  concept  to  higher  spatial  dimensions. 


2.2  Sensor  Fusion  Algorithms  over  Space,  Time,  and  Hierarchy 

The  research  described  in  this  section  deals  with  efficient  algorithms  for  large-scale 
optimal  estimation  and  is  reported  in  detail  in  [2,5,7,9-10,18,20-21,26-27, 29, 35-36,42- 
45,60,62-63,68].  The  general  objective  for  this  part  of  our  research  is  to  investigate 
stochastic  models  with  structure  that  can  be  exploited  in  order  to  develop  optimal  or  near- 
optimal  algorithms  that  are  scaleable  to  the  large-scale  problems  encountered  in  image 
analysis,  sensor  fusion,  and  higher-level  fusion.  In  our  work  we  have  demonstrated  the 
power  of  multiscale  linear/Gaussian  models  on  trees  in  terms  of  the  efficient  algorithms  that 
such  models  exist.  Our  most  recent  aimed  at  significant  extensions  of  the  class  of  models 


for  which  we  can  obtain  algorithms  that  are  as  powerful  or  nearly  as  powerful  as  those  for 
linear-Gaussian  models  on  trees. 

g)  The  predominant  algorithmic  concept  that  is  under  investigation  by  researchers 
in  the  field  of  graphical  models  is  Pearl’s  algorithm,  which  involves  utilization  only  of  the 
local  structure  of  a  graph,  ignoring  global  structure  such  as  loops.  While  some 
convergence  results  (both  positive  for  estimates  and  negative  for  error  covariances)  exist, 
there  is  a  fundamental  limitation  to  this  approach,  namely  its  exploitation  only  of  local 
structure.  Thus,  in  particular,  one  might  expect  this  algorithm  to  have  the  same  slow 
convergence  properties  as  Gauss-Seidel  algorithms  for  large  linear  systems  corresponding 
to  standard  MRF  models  on  2-D  lattices  or  to  elliptic  PDE’s.  Our  very  recent  work  [44,60] 
is  aimed  at  exploiting  global  structure  explicitly.  Indeed,  this  is  exactly  the  point  of  our  tree 
algorithms.  That  is,  it  is  precisely  the  global  structure  of  trees  (i.e.,  the  absence  of  loops) 
that  allows  us  to  propagate  estimates  very  efficiently  throughout  the  entire  graph.  In  very 
recent  work  we  have  looked  at  the  question  of  what  happens  if  we  view  the  edge  set  of  a 
graph  with  loops  as  the  union  of  edge  sets  of  trees  each  of  which  comes  from  discarding  a 
subset  of  the  edges  of  the  original  graph.  Using  this  structure  we  can  directly  define 
iterative  algorithm  in  which  we  perform  tree-based  estimation  using  each  of  these 
embedded  tree  models  in  succession.  To  date  we  have  performed  experiments  which 
indicate  that,  at  least  for  certain  types  of  graphical  models,  this  approach  leads  to  far  more 
efficient  algorithms  for  estimation  and  for  accurate  computation  of  error  covariances. 

h)  We  have  recently  developed  a  new  approach  to  large-scale  estimation  that  adapts 
and  extends  so-called  Krylov  subspace  algorithms  for  solving  large  linear  systems 
[26,35,36,68].  The  key  to  the  efficiency  of  such  algorithms  in  applications  such  as  solving 
discretized  PDE’s  is  being  able  to  quickly  calculate  particular  matrix-vector  products.  In 
the  estimation  context  the  matrix  of  concern  is  the  data  covariance  matrix.  Our  contribution 
is  roughly  threefold.  The  first  is  demonstrating  the  applicability  of  Krylov  subspace 
methods  for  large-scale  spatial  and  space-time  estimation  for  nontrivial  classes  of  random 
field  models— in  fact  models  that  are  in  some  sense  complementary  to  those  for  which 
multiresolution  methods  are  directly  applicable.  The  second  contribution  represents  a 
nontrivial  extension  of  Krylov  subspace  algorithms  for  the  computation  of  quantities  that 
are  not  typically  of  interest  in  other  applications  such  as  solving  PDE’s  but  are  essential  in 
our  context,  namely  the  calculation  of  estimation  error  covariances.  Finally,  the  third 
contribution  is  the  development  of  a  first  set  of  convergence  results  for  our  algorithm. 

While  there  are  convergence  analyses  in  the  numerical  linear  algebra  literature,  the 
difference  in  our  work  is  that  we  needed  to  consider  the  fact  that  the  quantities  of  interest 
are  described  statistically  (and,  in  particular,  the  data  are  corrupted  by  noise). 

As  described  in  [26,68]  we  have  also  developed  an  extension  of  the  Krylov- 
subspace  algorithm  to  deal  with  large-scale  time-recursive  estimation  for  space-time 
processes.  In  particular,  the  basic  Krylov-subspace  algorithm  provides  a  fast  algorithm  for 
incorporating  measurements  assuming  that  certain  covariance-vector  products  can  be 
calculated  quickly.  In  a  recursive  estimation  context  that  algorithm  can  be  used  in  the 
measurement  update  step  assuming  both  that  we  can  propagate  the  required  error 
covariance  and  that  the  resulting  covariance  has  structure  that  leads  to  fast  matrix- vector 
products.  Achieving  these  two  properties  involves  the  development  of  a  second  Krylov- 
subspace  algorithm,  in  this  case  for  the  approximate  propagation  of  the  error  statistics  over 
time  as  the  temporal  dynamics  are  used  to  predict  estimates  to  the  next  measurement  update 
time.  This  is  a  nontrivial  task,  as  it  is  not  obvious  that  Krylov-subspace  and  conjugate 
gradient  methods  can  be  applied  to  the  problem.  However,  in  our  work  we  have  developed 
a  concept  for  doing  this  which  has  shown  its  promise  in  preliminary  testing. 

i)  We  have  developed  a  nonlinear/hybrid  image  analysis  framework  that  combines 
our  multiresolution  estimation  algorithms  with  the  estimation  of  “hidden  variables” 


modeling  data  anomalies  or  abrupt  changes  (e.g.,  edges  or  discontinuities)  in  the  scene 
being  imaged  [18,27,45].  The  key  to  this  extension  is  the  use  of  the  EM  algorithm, 
coupled  with  our  efficient  multiscale  estimation  algorithm,  in  order  to  reconstruct  or 
denoise  images  subject  to  these  anomalies  and  abrupt  changes.  The  methodology  for  this 
extension  has  now  been  developed,  and  we  are  nearing  completion  of  the  investigation  of 
its  application  in  a  prototypical  and  important  military  imaging  problem,  namely  the 
estimation  of  object  shapes  in  laser  radar  range  images.  As  with  SAR  and  ISAR,  such 
images  are  subject  to  significant  outliers  due  to  speckle,  and  we  have  demonstrated  that  we 
can  use  our  new  EM/Multiresolution  framework  not  only  to  account  for  these  anomalies  but 
also  to  accurately  separate  a  target  from  the  background  (by  modeling  the  abrupt  change  in 
range  or  range  slope  when  moving  from  background  to  target  pixels  as  other  hidden 
variables)  and  then  to  estimate  target  shape  robustly. 


2.3  Statistical  Modeling  and  Estimation  of  Shape  with  Applications  in 
Object  Extraction  and  Recognition 

The  research  described  in  this  section  represents  one  of  the  more  recent  and  most 
exciting  components  of  our  research  and  one  which  we  believe  has  substantial  further 
promise  for  the  future.  In  particular,  the  general  objective  of  this  part  of  our  research  is  the 
development  of  statistically  robust  methods  for  segmentation  and  shape  estimation  with 
applications  ranging  from  wide-area  mapping  to  object  recognition,  and  the  results  of  our 
efforts  in  this  area  are  developed  in  detail  in  publications  [11,12,18,25,30-32,39,46,50,55- 
57]. 


j)  In  recent  years  a  theory  has  developed  for  so-called  curve  evolution  equations  in 
which  the  objective  is  to  smooth  or  evolve  curves  in  images  in  order  to  find  object 

boundaries  or  segment  images.  One  rich  framework  for  doing  this  involves  the 

minimization  of  functionals  related  to  the  geometry  of  the  curves.  For  example,  minimizing 
curve  length  leads  to  a  curve  evolution  in  which  the  local  speed  of  "shrinking"  of  a  curve  is 
determined  by  the  curvature  at  each  point.  It  had  been  known  for  some  time  that  one  way  in 
which  to  apply  this  algorithm  to  segmentation  involved  the  use  of  some  image-guided 
metric  for  measuring  the  local  length  (and  hence  the  curvature)  of  a  curve.  In  particular,  if 
one  defines  a  local  measure  of  curve  length  in  terms  of  the  local  image  gradient  so  that 
curves  have  much  smaller  length  in  regions  of  high  image  gradient,  one  can,  in  principle, 
design  curve  evolutions  that  will  evolve  to  shortest  length  curves  (in  this  adapted  metric) 
that  correspond  to  edges  of  regions  or  objects.  Computing  image  gradients,  of  course,  is  a 
notoriously  noisy  process,  so  that  ad  hoc  methods  for  obtaining  smoothed  image  gradients 
were  used.  As  a  result,  these  algorithms  had  limited  robustness  and  tended  either  to  fail  for 
very  noisy  images  or  to  require  excessive  or  ad  hoc  smoothing  that  limited  the  sharpness  of 
the  resulting  segmentations.  Moreover,  the  lack  of  a  statistical  formulation  made  it  difficult 
to  understand  how  these  algorithms  might  be  modified  in  order  to  enhance  robustness. 
Furthermore,  the  evolutions  had  no  natural  way  in  which  to  grow  rather  than  shrink,  which 
required  either  a  very  carefully  chosen  initial  condition  for  the  curve  or  an  ad  hoc  initial 
period  of  curve  dilation  before  initiating  curve  shrinkage. 

This  state  of  affairs,  as  well  as  the  intellectual  objective  of  constructing  a  statistical 
framework  for  curve  evolution,  provided  the  motivation  for  our  recent  research.  The 
simplest  setting  in  which  to  understand  the  ideas  behind  our  first  work  in  this  area  is  the 
binary  segmentation  problem  in  which  we  wish  to  partition  an  image  into  two  subsets 
(neither  of  which  needs  to  be  connected~i.e.,  each  can  consist  of  an  unknown  number  of 
connected  components)  so  that  the  image  pixel  values  have  distinctly  different  mean  values 
in  each  region.  Equivalently,  we  need  to  find  the  curve  C  defining  the  boundary  between 
these  two  subsets  (so  that  tins  curve  itself  may  consist  of  an  unknown  number  of 
disconnected  subcurves).  The  statistical  optimality  criterion  we  use  to  estimate  C  is  simply 


the  negative  of  the  square  of  the  difference  of  the  sample  means  computed  in  two  regions 
separated  by  C  (i.e.,  inside  and  outside).  Gradient  descent  for  the  minimization  of  this 
functional  then  leads  to  a  remarkably  simple  and  robust  curve  evolution  equation.  In 
particular,  rather  than  using  local  gradients  to  evolve  the  curves,  this  new  evolution  uses 
nonlocal  and  thus  statistically  more  stable  statistics,  namely  the  sample  means  inside  and 
outside,  as  drivers.  The  resulting  evolution  is  quite  robust  to  noise  and  in  particular 
produces  remarkably  accurate  segmentations  of  images  even  when  starting  from  what  are 
essentially  arbitrary  initial  curves  consisting  of  small  subcurves  distributed  over  the  entire 
image  domain.  We  have  also  shown  how  this  method  extends  to  the  segmentation  of  an 
image  into  more  than  two  statistically  distinct  regions  which  are  not  necessarily  required  to 
be  disjoint  (a  potentially  useful  capability  if  occlusion  is  a  significant  issue). 

k)  In  the  second  component  of  our  work  in  this  area  we  have  adapted  the  flow 
described  in  (j)  by  including  a  global  constraint  that  significantly  aids  performance  for 
low-contrast  images  with  complex  region  shapes.  In  particular,  the  evolutions  defined  by 
our  work  in  (j)  move  a  curve  dynamically  in  order  to  increase  the  squared-difference 
between  the  sample  means  inside  and  outside  of  the  curve.  Since  the  objective  is  simply  to 
increase  the  squared-difference,  it  is  perfectly  possible  for  the  curve  to  evolve  so  that  the 
means  move  in  the  same  direction  (e.g.,  both  increase)  but  in  a  manner  such  that  their 
difference  also  increases  in  magnitude.  By  modifying  the  evolution  it  is  possible  to  ensure 
that  the  curve  evolution  always  moves  not  only  to  increase  the  difference  in  means  but  also 
to  drive  each  of  the  means  in  opposite  directions.  This  modification  does  not  affect 
computational  complexity  of  the  algorithm  at  all  but  produces  rather  remarkable 
segmentation  for  very  complex  images  (e.g.,  for  biomedical  images  of  various  types). 

l)  In  [46]  we  describe  research  stemming  from  a  curve-evolution  interpretation  and 
solution  of  a  widely  known  combined  image  denoising  and  segmentation  problem 
introduced  by  Mumford  and  Shah.  This  extension  not  only  is  of  great  intellectual 
significance  in  showing  an  explicit  tie  between  curve  evolution  and  statistical  image 
analysis  but  also  leads  to  some  very  interesting  and  important  new  image  processing 
algorithms  In  particular,  our  analysis  has  established  that  the  solution  to  this  problem 
involves  a  curve  evolution  coupled  with  the  solution  of  elliptic  PDE’s  inside  and  outside  the 
curve  These  PDE’s  characterize  the  optimal  image  estimate  in  each  region  based  on  the 
current  estimated  location  of  the  curve.  The  behavior  of  these  estimates  near  the  current 
curve  location  then  provide  the  driving  term  for  the  curve  evolution,  generalizing  in  a  very 
elegant  manner  the  work  described  in  (j)  in  which  the  sample  means  inside  and  outside  are 
used  together  with  image  values  on  either  side  of  the  current  curve  location  to  drive  the 
curve  evolution.  As  can  be  seen  in  [46],  the  resulting  algorithm  has  remarkable  capabilities 
for  segmenting  images  into  multiple  regions  automatically  without  prior  knowledge  that 
there  are  multiple  regions  (or  how  many  regions  might  be  present).  Moreover,  because  of 
its  estimation-theoretic  basis,  the  algorithm  works  equally  well  if  there  are  missing  data  or 
data  gaps  as  can  occur  for  surveillance  imagery  (e.g.,  due  to  speckle,  shadowing,  or 
blockage  due  to  cloud  cover).  In  fact,  using  this  same  idea,  we  can  extend  this  method  to 
perform  image  magnification:  we  view  our  observed  image  as  a  subsampled  version  of  a 
finer-scale  image  and  then  use  our  algorithm  to  smoothly  interpolate  to  the  higher  resolution 
both  inside  and  outside  the  curve  without  any  blurring  across  the  curve  boundary.  Our 
results  demonstrate  that  this  approach  is  far  superior  to  any  previously  known  method  for 
image  magnification. 


2.4  Blending  Physics  and  Statistical  Learning  for  Image  Reconstruction, 
Feature  Extraction,  and  Fusion 


In  this  section  we  describe  our  research  on  marrying  sensor  physics  with  statistics 
in  order  to  develop  robust  methods  for  exploiting  the  information  present  in  sensor  data. 

The  fundamental  idea  here  is  that  a  full,  heavy  use  of  fundamental  physics  (e.g.,  solving 
Maxwell’s  equations)  is  clearly  inappropriate  for  sensor  fusion  since  there  are  limitations  in 
the  “apertures”  through  which  we  view  the  phenomenon  both  from  the  input  and  output 
sides.  In  particular,  the  data  that  are  typically  available  (e.g.,  to  form  a  SAR  image)  are  far 
too  limited  in  extent  and  subject  to  too  many  sources  of  uncertainty  and  variability  to 
warrant  full  solution  of  the  inverse  scattering  problem  in  order  to  form  an  image. 

Fortunately,  and  complementary  to  the  limitations  in  the  observed  data  is  the  fact  that  the 
desired  information  we  wish  to  extract  from  such  sensor  data-e.g.,  detections  and 
classifications  of  objects — are  far  more  limited  than  a  complete  inverse  scattering  solution. 

On  the  other  hand,  discarding  all  physics  also  is  unwise,  as  the  constraints  implied  by 
sensor  physics  can  be  used  to  reduce  the  apparent  number  of  degrees  of  freedom  in  the 
data,  thus  enhancing  the  quality  of  any  statistical  analysis.  The  challenge  is  to  determine 
the  proper  balance  between  physics/model-based  methods  and  statistical/leaming  methods. 
This  is  a  deep  and  enduring  theme  to  which  we  believe  we  have  made  some  contributions. 
Our  work  to  date  in  this  broad  area  is  described  in  [4,8,10-11,23-24,37-38,40-41,48- 
49,51,53-54,58,65-66,69]. 

m)  We  have  investigated  a  first  principles  estimation-theoretic  approach  to 
Synthetic  Aperture  Radar  (SAR)  imaging  of  moving  scenes.  In  particular,  we  have 
extended  the  general  SAR  image  formation  problem,  which  aims  to  estimate  radar  cross 
section  (RCS)  in  each  pixel,  to  allow  for  the  joint  estimation  of  both  RCS  and  a  scene 
velocity  vector  in  each  pixel.  However,  the  maximum  likelihood  estimation  problem  for 
the  reconstruction  of  both  of  these  quantities  is  a  highly  ill-posed  variational  problem  unless 
we  incorporate  a  regularizing  prior  model  either  on  RCS  variations  or  scene  velocity  (or 
both)  across  the  image  domain.  The  approach  we  have  taken  to  addressing  this  problem  is 
that  of  formulating  variational  problems  corresponding  to  the  joint  MAP  estimation  of  RCS 
and  velocity  fields  over  the  scene  domain.  In  particular,  to  capture  the  fact  that  we  want  to 
final  image  to  be  sharply  focused,  we  have  investigated  the  use  of  an  penalty  on  the 
reconstructed  RCS  field.  As  Donoho  has  thoroughly  discussed,  the  inclusion  of  such  a 
penalty  tends  to  favor  sparse  representations  which  in  this  case  corresponds  to  the 
reconstruction  of  sharp  images  of  bright  scatterers.  Interestingly,  this  term  by  itself  leads 
to  remarkably  enhanced  SAR  reconstructions  even  when  we  know  that  there  is  no  motion 
in  the  scene  being  imaged.  In  particular  the  presence  of  this  sharpening  term  suppresses 
standard  SAR  sidelobes  dramatically.  Moreover,  this  regularizing  term  by  itself  allows  us 
to  estimate  coarse  scene  motion-e.g.,  the  mean  velocity  over  a  target  region  is  accurately 
estimated  simply  with  the  inclusion  of  this  L,  penalty  on  the  reconstructed  RCS  field  and 
without  any  regularization  of  the  velocity.  However,  as  we  begin  to  allow  for  varying 
velocities  across  the  image  scene  (as  is  the  case  in  typical  SAR  and  ISAR  imaging  scenarios 
due  to  non-translational  and  nonrigid  target  motion),  our  preliminary  results  make  it  clear 
that  we  will  need  to  introduce  a  regularizing  prior  for  the  velocity  field  as  well. 

n)  A  second  component  of  our  research  in  this  area  involves  the  development  of  a 
methodology  that  combines  SAR  scattering  physics  with  observed  SAR  imagery  taken  at  a 
number  of  different  aspects  to  construct  a  SAR  target  model  that  captures  the  salient 
features  that  are  robustly  extractable  from  SAR  data.  Rather  than  taking  the  mathematical 
physics  perspective  of  viewing  this  as  a  very  ill-posed  inverse  scattering  problem,  we  take 
advantage  of  the  fact  that  salient  features  for  S  AR-based  target  discrimination  are  generally 
confined  to  a  much  lower-dimensioned  set,  corresponding  to  significant  scatterers  of 
different  types.  In  our  initial  work  in  this  area  we  have  begun  by  limiting  our  attention  to  a 
small  number  of  discrete  scattering  mechanisms,  each  of  which  has  a  well-defined, 
deterministic  scattering  response.  The  idea  then  is  that  individual  SAR  images  are 
processed  in  order  to  extract  estimated  locations  and  types  of  features.  Processing  of  these 


individual  images,  however,  results  in  the  extraction  of  false  features,  missed  detections  of 
some  actual  features,  and  the  misclassification  of  others  due  to  noise,  clutter,  and  to  the 
limited  aperture  used  in  each  individual  SAR  image.  However,  by  associating  features 
from  multiple  images  using  statistically  optimal  methods,  we  can  obtain  a  consistent  image 
that  rejects  spurious  detections  in  individual  SAR  images  and  corrects  for  misses  and 
misclassifications.  Our  results  demonstrate  the  potential  of  this  approach  and  also  suggest 
ways  in  which  to  enhance  it  significantly  and,  more  fundamentally,  to  understand  more 
deeply  how  to  exploit  sensor  physics  in  a  statistically  significant  manner. 

In  particular,  we  have  also  taken  a  first  set  of  steps  aimed  at  relaxing  the  restriction 
on  scattering  types  in  order  to  capture  some  of  the  variability  of  scattering.  Specifically,  one 
of  the  important  characteristics  that  is  certainly  captured  in  the  full  inverse  scattering 
formulation  but  is  lost  if  we  insist  on  compressing  each  individual  SAR  view  to  a  small, 
discrete  set  of  types  is  the  characteristic  variability  of  the  scattering  response  from  a 
distinctive  scatterer  when  we  look  at  it  from  multiple  viewing  angles.  If  we  could  exploit 
that  characteristic  signature  across  views  we  would  undoubtedly  obtain  better 
reconstructions  of  SAR  scattering  features  and,  in  particular,  a  better  indication  of  the 
robust  and  salient  information  that  can  be  extracted  from  SAR  images.  In  our  initial  work 
we  have  taken  an  initial  and  very  limited  look  at  how  we  might  extend  our  framework  to 
capture  some  of  this  variability  for  a  specific  scattering  mechanism.  In  particular,  some 
scatterer  types  (cylinders  and  so-called  “top  hats”)  appears  as  bright  spots  at  different 
positions  in  images  taken  at  different  views  due  to  the  nonzero  radius  of  the  scatterer.  This 
can  lead  to  biases  in  the  estimates  of  the  location  of  the  scatterer  and  to  incorrect  rejection  of 
the  feature  if  this  characteristic  is  not  recognized.  However,  if  we  include  scatterer  radius 
as  another  parameter  to  be  estimated  as  part  of  the  process  of  fusing  information  from 
multiple  views,  we  can  correct  for  this  bias  and  correctly  associate  features  extracted  from 
different  views  that  otherwise  would  appear  to  be  originating  from  different  locations. 

o)  Building  on  earlier  work  on  multiresolution  analysis  of  SAR  imagery,  we  have 
taken  a  deeper  look  marrying  SAR  physics  with  nonparametric  statistical  learning  methods 
for  constructing  probabilistic  models  for  multiresolution  imagery.  In  particular  consider 
the  formation  of  SAR  imagery  based  on  a  given  full  aperture  of  data.  If  we  use  the  entire 
aperture,  we  obtain  imagery  at  the  finest  resolution  resolvable  using  that  data.  However,  to 
do  this  we  in  essence  must  assume  that  all  scattering  is  isotropic,  i.e.,  that  the  response 
from  significant  scatterers  is  constant  across  the  entire  aperture.  For  many  important 
scattering  mechanisms  this  is  not  the  case  at  all,  and  this  anisotropy  is  critical  to 
distinguishing  one  scatterer  type  from  another.  Suppose  then,  that  in  addition  to  forming 
an  image  using  the  entire  aperture,  we  also  form  three  images  each  using  half  of  the 
aperture:  one  image  using  the  right  half,  one  the  left,  and  one  using  a  centered  half¬ 
aperture!  If  indeed  there  are  anisotropic  scatterers,  we  might  expect  that  there  would  be 
differences  in  the  responses  in  each  of  these  half-apertures  and  hence  in  the  images  formed 
using  them  (note  that  these  images  would  have  pixel  sizes  twice  as  large  as  the  ones  in  the 
finest  scale  imagery).  Iterating  this  process,  we  can  imagine  forming  a  vector  of  images  at 
each  of  a  sequence  of  scales  corresponding  to  progressively  smaller  subapertures.  By 
looking  across  scale,  then,  we  would  expect  not  only  to  find  statistical  variability  due  to 
speckle  but  also  any  evidence  of  anisotropic  scattering  manifesting  itself  in  statistically 
significant  differences  in  pixel  intensities  in  images  formed  using  different  subapertures. 

Characterizing  these  differences  in  a  statistically  sound  manner  so  that  they  can  be 
exploited  either  for  the  extraction  of  features  or  for  the  recognition  of  objects,  however, 
requires  a  significant  departure  from  the  linear-Gaussian  modeling  framework  for 
multiresolution  modeling  on  which  we  have  focused  most  of  our  effort  in  the  past.  In 
particular,  we  now  are  faced  with  a  multiresolution  vector  pyramid  of  images  where  the 
dimensionality  of  the  vector  is  larger  at  coarser  resolution  (since  forming  a  coarser 
resolution  image  requires  a  commensurably  smaller  subaperture  of  the  SAR  data,  allowing 
more  images  to  be  formed).  Furthermore  and  most  importantly,  much  as  in  (d),  the 


behavior  across  scale  in  this  multiresolution  pyramid  is  decidedly  non-Gaussian  (as,  in 
particular,  we  are  especially  interested  in  significant  outliers  corresponding  to  characteristic 
behavior  of  particular  anisotropic  scatterers).  In  our  work  along  these  lines  we  have 
investigated  the  use  of  nonparametric  density  estimation  techniques  to  build  models 
capturing  statistical  variability  from  scale  to  scale. 

p)  Finally,  in  some  very  recent  research  we  have  explored  the  use  of  nonparametric 
statistics  and  statistical  learning  methods  for  problems  of  fusing  information  from 
completely  different  sensing  phenomenologies.  In  doing  this  we  have  used  several  very 
different  applications  as  vehicles.  The  first  of  these  involves  so-called  functional  MR 
imaging  (fMRI).  In  this  application,  a  patient  performs  a  specific  protocol,  such  as 
squeezing  a  ball,  waving  an  arm,  listening  to  words,  speaking  words,  etc.,  while  at  the 
same  time  a  sequence  of  MR  images  is  taken  of  his  or  her  brain.  The  challenge  of  fMRI, 
then,  is  to  determine  which  voxels  in  a  3-D  MR  image  volume  involve  activity  in  response 
to  the  protocol.  Looking  at  a  single  candidate  voxel,  we  have  the  problem  of  determining  if 
there  is  any  relationship  between  two  time  signals,  one  being  the  protocol  (typically  an  on- 
off  square  wave  signal,  e.g.,  corresponding  to  squeezing  a  ball  and  then  relaxing  in  a 
periodic  manner)  and  the  other  being  the  time  history  of  the  MR  image  in  the  candidate 
voxel.  While  standard  correlation  methods  (corresponding  to  assuming  a  linear 
relationship  between  these  signals)  are  sometimes  used,  it  is  clear  on  looking  at  the  signals 
that  the  relationship  is  highly  nonlinear.  As  a  result,  we  have  the  question  of  determining 
if,  in  some  precisely  measurable  way,  the  information  in  these  two  signals  is  related.  The 
approach  we  have  taken  is  to  use  mutual  information  as  a  metric  to  quantify  this 
relationship  and  have  adopted  the  use  of  nonparametric  statistical  methods  in  order  to 
estimate  this  quantity.  At  this  point  we  have  preliminary  and  very  promising  results  on  this 

application.  „  . ,  .  . ,  .  .  , 

-  The  second  application  involves  the  fusion  of  video  imagery  with  acoustic  signals 
sensed  by  microphones.  In  this  case,  the  objective  is  to  determine  what  action  in  the 
viewed  scene  is  the  source  of  the  sensed  acoustic  signal  (or  what  actions  are  responsible  for 
the  acoustic  signal  if  there  are  several  acoustic  sources  to  be  separated).  We  have 
developed  a  nonparametric  statistics  approach  to  finding  the  linear  projection  of  the  image 
scene  that  has  maximal  mutual  information  with  the  sensed  acoustic  signal.  The  results  on 
preliminary  imagery  are  striking  in  their  ability  to  localize  sources  of  sound. 
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V.  INTERACTIONS/TRANSITIONS 


In  this  section  we  summarize  the  various  interactions  and  transitions  associated  with 
research  supported  by  AFOSR  Grant  F49620-98- 1-0349,  focusing  on  the  last  year  of  this 
grant  (we  refer  the  reader  to  previous  progress  reports  for  information  on  previous 
activities  in  these  areas). 

Participation/Presentation  at  Meetings 

In  addition  to  presentations  at  professional  conferences,  we  have  been  involved  in 
the  following  other  meetings  during  the  past  year: 

(1)  Prof.  Willsky  has  continued  to  have  regular  meetings  with  engineers  and 
researchers  at  Alphatech  to  discuss  collaborative  research  in  the  area  of  SAR-based  ATR 
and  more  recently,  in  multiresolution  modeling  and  fusion  for  global  awareness  in 
connection  with  an  Alphatech  project  under  AFOSR's  New  World  Vistas  initiative.  In 
addition,  our  work  on  graphical  models  has  direct  relevance  to  DARPA’s  Dynamic 
Database  Program,  and  we  have  initiated  discussions  with  Alphatech  on  topics  of  mutual 

interest. 

(2)  In  January  2000  Prof.  Willsky  delivered  the  keynote  address  at  the  AFRL- 
AFOSR  Workshop  on  Future  Directions  in  Sensor  Fusion  and  Automatic  Target 
Recognition,  held  at  Eglin  AFB. 

(3)  At  the  request  of  AF  LGen  (Ret.)  Lincoln  Faurer,  President  of  the  National 
Correlation  Working  Group  (and  former  Director  of  the  National  Security  Agency),  Prof. 
Willsky  participated  as  one  of  the  lead  panelists  on  data  and  sensor  fusion  at  the  May  2000 
NCWG  Workshop  on  Information  for  the  Warfighter,  held  at  Fort  Monroe,  Virginia,  m 
collaboration  with  the  Aerospace  C2ISR  Center  in  Langley. 


Consultative  and  Advisory  Functions 


During  the  year,  Prof.  Willsky  has  been  engaged  in  the  following  activities  relevant 
to  the  research  being  performed  under  our  AFOSR  grant: 


(1)  Prof.  Willsky  has  regularly  acted  as  a  consultant  to  Alphatech,  Inc.  in  a  number 
of  research  projects  including  ones  that  represent  direct  transitions  of  the  technology  being 
developed  under  our  AFOSR  Grant. 

(2)  Over  the  past  2.5  years  Prof.  Willsky  has  acted  as  co-chair  (with  Dr.  Wendy 
Martinez  of  ONR)  of  a  tri-service  working  group  on  the  role  of  probability  and  statistics  in 
command  and  control.  Prof.  Willsky  was  the  principal  author  of  the  report  produced  by 
this  panel. 


(3)  In  October  1998,  Prof.  Willsky  was  appointed  to  the  Air  Force  Scientific 
Advisory  Board.  During  his  first  year  in  this  position.  Prof.  Willsky  participated  in  the 
S&T  Review  of  AFRL/SN  and  the  relevant  parts  of  AFOSR  supporting  SN,  and  he  also 
was  an  active  participant  on  the  Intelligence  and  Vigilance  Panel  for  the  1999  AF/SAB 
Summer  Study  on  Technology  Options  to  Leverage  Aerospace  Power  for  Operations  Other 
than  Conventional  War.  During  the  Fall  of  200,  Prof.  Willsky  served  on  the  S&T  Review 
Panel  for  AFRL/IF  and  the  associated  AFOSR  programs,  and  he  has  recently  completed  his 
effort  as  a  member  of  the  Technology  Panel  in  this  year’s  AF/SAB  Summer  Study  on 


AFC2:  The  Way  Ahead  (a  topic  specifically  requested  by  the  Air  Force  Chief  of  Staff), 
including  being  the  principal  author  of  recommendations  on  sensor  and  data  fusion. 

(4)  At  the  request  of  Mr.  E.  Zelnio  of  AFRL/SN,  Prof.  Willsky  participated  as  a 
member  of  an  ad-hoc  panel  helping  Mr.  Zelnio  and  AFRL  with  its  plan  for  technology 
insertion  to  meet  both  short-  and  intermediate-term  objectives  related  to  the  ‘Tanks  under 
Trees”  initiative  requested  by  the  Air  Force  Chief  of  Staff  in  response  to  needs  identified  in 
Kosovo  operations. 

Transitions 

The  following  are  the  transitions  of  our  research  that  are  taking  place: 

(1)  Our  multiresolution  SAR  discrimination  algorithms,  most  recently  for  the 
classification  of  nonisotropic  scattering  behavior,  are  being  transitioned  to  Alphatech  for 
inclusion  in  advanced  model-based  ATR  algorithms.  The  points  of  contact  on  this  are  Dr. 
Robert  Washburn,  Dr.  John  Wissinger,  and  Dr.  Gil  Ettinger. 

(2)  Our  work  on  multiresolution  SAR  analysis  and  high-resolution  pursuit  have 
been  transitioned  to  Alphatech  under  a  Small  Business  Technology  Transfer  contract  from 
the  Army  Research  Office  to  transition  our  multiresolution  mapping  and  estimation  methods 
to  military  applications.  The  points  of  contact  at  Alphatech  for  this  are  Dr.  Robert 
Washburn  and  Mr.  Thomas  Allen. 

(3)  Our  efficient  methodology  for  multiresolution  mapping  and  data  fusion  are 
being  transitioned  to  Alphatech  as  part  of  an  SBIR  program,  through  NIMA,  on  fusion  of 
multiresolution  and  multipass  data  to  produce  high-fidelity  terrain  maps.  The  point  of 
contact  is  Mr.  Thomas  Allen. 

(4)  Our  current  work  on  building  SAR  scattering  models  from  multiple-view  SAR 
imagery  is  being  carried  out  in  close  contact  with  Alphatech  engineers  to  ensure  rapid 
transition.  The  points  of  contact  are  Dr.  John  Wissinger  and  Dr.  Gil  Ettinger. 

(5)  Our  work  on  stabilized  inverse  diffusion  equations  for  nonlinear  image 
segmentation  has  been  transitioned  to  Alphatech  for  the  segmentation  of  IR  imagery  as  part 
of  an  AFRL  Sensors  Directorate  program  on  SAR-FLIR  fusion.  The  point  of  contact  on 
this  is  Mr.  Thomas  Allen. 

(6)  We  have  recently  initiated  discussions  with  Alphatech  on  the  transition  of  our 
work  on  statistical  image  segmentation  and  region  estimation  using  curve  evolutions,  both 
to  DARPA’s  Dynamic  Database  (DDB)  Program  and  to  imaging  projects  in  medical 
imaging  and  video  surveillance.  The  points  of  contact  on  this  are  Dr.  Joel  Douglas,  Dr. 
John  Wissinger,  and  Dr.  Gil  Ettinger. 

(7)  We  have  also  begun  transitioning  our  most  recent  work  on  covariance  extension 
and  graphical  models  to  an  Alphatech  SBIR  on  road  network  estimation,  an  adjunct  to 
DARPA’s  DDB  Program.  The  point  of  contact  is  Dr.  Joel  Douglas. 


